The present invention relates to an image processing technology for reducing image shaking caused by shaking or motion of an image pickup apparatus.
As an image stabilization method for reducing image shaking accompanying shaking and motion (hereinafter, collectively referred to as “motion”) of the image pickup apparatus, there is electronic image stabilization which uses an image processing technology. The electronic image stabilization includes one that performs image stabilization processing which calculates motion vectors for frame images sequentially generated by the image pickup apparatus, and then shifts or deforms an actually outputting area (clipping area) of each frame image based on the motion vectors.
In such electronic image stabilization that uses the motion vectors, however, if a moving object such as a pedestrian or a vehicle is present within an image pickup area, not only a motion vector by motion of the image pickup apparatus but also a motion vector by motion of the moving object are calculated. Then, if the image stabilization processing is carried out based on the motion vector of the moving object, image shaking accompanying the motion of the image pickup apparatus cannot be sufficiently reduced.
Thus, in an electronic image stabilization method disclosed in Japanese Patent Laid-Open No. 4-180370, plural detection areas is provided in each frame image, and a motion vector is calculated for each detection area. Then, the detection areas having small differences in direction and size of the detected motion vectors are collected into one group (cluster), which is referred to as “clustering processing”. In this way, a cluster of the motion vectors by the image pickup apparatus and a cluster of the motion vectors by the moving object are separated from each other.
Then, image stabilization processing is performed based on the motion vectors included in the cluster of the motion vectors by the motion of the image pickup apparatus.
In the electronic image stabilization method disclosed in Japanese Patent Laid-Open No. 4-180370, a cluster present before the moving object enters the image pickup area is selected as a cluster of motion vectors by the motion of the image pickup apparatus (hereinafter, referred to as an image stabilization target cluster meaning a cluster including image stabilization target motion vectors).
According to such an electronic image stabilization method, when an amount of motion of the image pickup apparatus is small due to, for example, hand jiggling, fluctuation is limited in positions of the image stabilization target clusters in the sequentially generated frame images. As a result, good image stabilization processing can be continued without losing the image stabilization target cluster.
However, when the amount of the motion of the image pickup apparatus is large or when a large camera motion such as panning or tilting is intentionally provided to the image pickup apparatus, great fluctuation occurs in positions of the image stabilization target clusters of the sequentially generated frame images. Thus, it is highly likely that the image stabilization target clusters are lost to cause a difficulty of continuing good image stabilization processing.